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I. INTRODUCTION 


As networks and the Internet become standard tools in manager’s toolkits, the 
usability of the applications’ interfaces will be a critical factor in their success. The 
American Heritage Dictionary defines usability as “in a fit condition for use,” with use 
being defined as “the application or employment of something for some purpose.” The 
term usability has been used to describe the way in which an application presents itself to 
the user. 

Users want to get their information quickly. They like Sellesteanieed sites that 
make information easy to find. (Morkes and Nielsen, 1997) Organizations must build 
sites that are more than status symbols. Most users are under some time constraint, either 
real or personally imposed. Two important reaeore users are willing to use applications 
are 1) to gain information from data and 2) the way the information is presented to the 
user. This information must provide some benefit to the user and it must be presented in 
an easy to understand and intuitive way. 

The information presented to any user comes from data that is distilled and 
analyzed for trends or patterns. This data must come from some repository, which in 
most cases is a database. Database management systems must have the ability to 


communicate with applications to aid in automation of provision of data to be analyzed. 


A. OBJECTIVE 


This thesis develops guidelines for building Web sites that are useful to senior 
managers in two ways. First, they can glean information from, or pass information to the 


site in order to accomplish their tasks better and more efficiently. Second, the senior 


manager should be able to go to a Web site and intuitively use that site without having to 
read a manual or obtain instruction before attempting to use that site. These managers 
can make better use of their time than needlessly searching manuals or receiving 
instruction. This technology should be in place to help these managers perform at a 
higher level. Furthermore, they will not resent using this technology to pull useful 
information from the Internet or intranet. 

This thesis focuses on oe main areas: the first being usability and design of a 
Web Site, and secondly, connecting database management systems (DBMS) and Web 
based applications. Together, these ideas enhance a Web site’s capability to become a 


more useful tool for everyone involved with its use. 


B. RESEARCH QUESTIONS 
Primary research questions include: 
1. What are the issues involved with building senior manager interfaces? 


e What is “user friendliness” — both from the senior manager’s standpoint and 
the site builders stand point? 


e Why should site builders/administrators be concerned with user friendliness? 
e What are the lessons learned from this experiment? 


e Can this study transcend boundaries to other Navy managers, or private 
industry CEOs and managers? 


2. What are the issues with respect to connecting an ODBC database to a Web 
page to provide the data to the interface, once it is properly designed? 


e What are the technologies for linking an ODBC Database to a Web page? 
— Web tool 
— Database tool 
— Connecting or communication tool 
— Where are these technologies moving? (i.e. what 1s on the horizon’) 


e Can a useful'(i.e. automated) Web page be created without the use of browser 
“plug-ins” for those using the Web page from outside the organization? 


C. SCOPE AND LIMITATIONS 


The scope of this thesis on interface issues for senior managers focuses on 
analyzing and developing the Leading Change in the Information Age (LCIA) Web site 
with usability in mind, and analyze using these techniques in a real world scenario. The 
scope on databases is to identify technologies most useful in linking databases with web 
pages. A case study is used to describe the current state of the LCIA Web site to test 
senior Management users in an attempt to profile their behaviors in relation to computer 
usage. Information gathered will be used to suggest guidelines for creating applications 
which they are more willing to use, and which are more intuitive. Qualitative rather than 
quantitative methods are appropriate for this project, since the number of participants in 
the study was limited to nine. Little literature exists relating senior managers and 
usability. Therefore, surveys involving direct interaction, interview, and observation 
were given to assess the interaction with technology. 

This study is important because of the lack of previous data collection and 
analysis in this context. Participants in the study were students in the July/August LCIA 
class. The researcher made every effort to construct survey questions that were neutral, 


SO as not to bias responses. 


D. THESIS ORGANIZATION 


This thesis consists of five chapters. Chapter II provides a background of the 


organization used in the case study, and provides explanation of the need for such a 


study. Chapter III presents the methodology used to obtain study data and information 
for this project. Chapter IV describes the study and findings relating to the use of ODBC 
databases and Web sites and findings on usability in relation to the responses of the 
senior managers using the LCIA site. Chapter V provides a summary, conclusions, and 
recommendations for areas of future research and suggestions for increasing the usability 


of the LCIA site. 


Il. BACKGROUND 


How many times have you seen computer systems sitting on a desk, in the back 
corner buried under a mountain of paper and dust? Upon inquiry as to the reason for the 
non-use of the system, the answer is usually, “because we were not trained on the system 
and we can not figure it out ourselves.” Systems can be so difficult and cryptic to 
operate, that users are repelled from using the system, and the organization has wasted its 
money on a highly priced paperweight. Users often give up on trying to use a system 
because of the arcane interface and cantankerous nature of the system. 

Senior managers are no different. Senior managers do not have the time to go 
through training courses to learn a system, because that is generally not their primary 
function. Senior managers use computer systems to transform complex data into usable 
information so they can aie intelligent decisions about how to perform their role, or to 
enhance the organization’s mission. Systems such as decision support systems and 
executive support systems or arene information systems can make these decisions in 
a more informed way, and much quicker. But these systems will not help anyone if they 
are not utilized. Therefore, systems must be designed with all users in mind. 

In the same vein, systems will certainly not be utilized if they do not present 
accurate or pertinent information. Information presented in these interfaces must be 
rooted in fact. These facts often come in the form of raw data, which is usually contained 
in a database. OBDC addresses the issue of letting the user application communicate 


with the database quickly and easily, without requiring a programmer to code each 


individual Web based application. This can severely impact the ease of use and 


attractiveness of these applications. 


A. THE CENTER FOR EXECUTIVE EDUCATION 


The Center for Executive Education (CEE) 1s part of, and located at, the Naval 
Postgraduate School in Monterey, California. The courses taught here are intended to 
address senior defense managers’ needs relating to the expansion of existing ties with 
industry and other organizations and institutions, and to study emerging technologies, 
practices, and policies in order to enhance the capabilities of DoN/DoD systems, 
commands, people, and information. The population the Center is geared to serve 
consists of military Officers with the ranks of O-7 through O-9, and their civilian 
executive counterparts. The center is “exclusively dedicated to enhancing the knowledge 
and understanding of leaders within the Department of the Navy (DoN) and Department 
of Defense (DoD) who will be making critical decisions affecting the nation’s readiness 
in this complex and rapidly changing environment” (CEE, 1998). To provide this 
educational experience, the center provides executive level education programs that 
‘furnish the tools and skills necessary to add value to senior defense leaders” (CEE, 
1998). The Center offers a variety of courses including: Leading Change in the 
Information Age (LCIA), Revolution in Business Practices (RBP), Topsail, and the Chief 
Information Officer Program. The latter two of these programs are under development at 
the time of this writing. 

This thesis uses the LCIA program as its test bed. The focus of the LCIA 


program is to increase the problem solving ability of senior defense executives in 


performing their leadership roles in information superiority. The goal is to make 
participants more familiar with: 
e Development and maintenance of sound integrated technology infrastructures 
and application systems 
e Application delivery and operation 
e Tools and methods used to monitor and evaluate performance of installed 
warfare systems 
e Resources and how to use them for optimal effect 
The material is concentrated on the underlying principles of technologies and on 
the ability to analyze and synthesize effective and flexible strategies. The intellectual 
framework consists of two components: 
e An architectures, systems, and technologies component, and 
e An information policy, strategy, evaluation, and operations component. 
Both components aim to provide a comprehensive understanding of information. (CEE, 


1998) 


B. THE LEADING CHANGE IN THE INFORMATION AGE PROGRAM 


The LCIA Program specifically serves O-7 and O-8 Officers and their civilian 
equivalents. The second course for 1998 was conducted at the Center from 26 July 
through 14 August. Parts of the curriculum for this program are designed to familiarize — 
the participants with current technologies in order to assist them in their jobs. The 
Internet and the World Wide Web, or simply the Web, are used to full advantage to help 


these people gather information on a myriad topics related to their jobs. Topics of 


interest to these individuals are very diverse, ranging from political to technical, and 
anything in between. The course helps participants learn how to access and filter the 
deluge of information available on the Web today. The Internet can be useful for 
gathering data from databases to be fed into applications, such as decision support 
systems (DSSs), used to create information deemed useful to making decisions for the 


future. 


C. INTERFACE DESIGN 


1. Reading on the World Wide Web 


In order to design interfaces geared toward Web based clients, one must first 
study how users read from that medium. According to the Nielsen Norman Group, "User 
experience" encompasses all aspects of the end-user's interaction with the organization, 
its services, and its products. 


The first requirement for an exemplary user experience is to meet 
the exact needs of the customer, without fuss or bother. Next comes 
simplicity and elegance that produce products that are a joy to own, a Joy 
to use. True user experience goes far beyond giving customers what they 
say they want, or providing checklist features. In order to achieve high- 
quality user experience in a company's offerings there must be a seamless 
merging of the services of multiple disciplines, including engineering, 
marketing, graphical and industrial design, and interface design. (Nielsen 
Norman Group, www.nngroup.com/about/userexperience.html, August 
14, 1998) 


These statements can easily be applied to any government scenario, where the customer 1s 
analogous to any user of an organization’s Web site and references to the company equate 


to the organization using a Web site for constructive purposes. 


A study conducted by John Morkes and Jakob Nielsen found that 79 percent of 
test users scanned any new page they came across, with only 16 percent reading word- 
for-word (Nielsen, 1997a). Four possible reasons for this particular behavior are 
(Nielsen, 1997b): 

e Reading from computer screens is tiring and approximately 25 percent 

slower than reading from paper; 

e The Web is a user-driven medium, 1.e. users feel they must move on 

and click on things; 

e Each page has to compete with literally millions of other pages for the 

user’s attention; 

e Modern life is hectic. 

According to research by Nielsen and others, individuals do not want to invest too much 
time in any one page while searching for information on the Web because the page may 
not contain the needed information. Thus, the page is quickly scanned for relevance in a 
quick once over before clicking on to the next page. 

Other important issues are the credibility of the information, since there is no 
guarantee about who has provided the information posted to the Web site. Studies by 
Nielsen have shown that by providing hypertext links to external related sites, using high 


quality graphics, and employment of good writing practices, credibility or perceived 


credibility of the information on a site can be significantly increased. 


2. Writing for the World Wide Web 


By understanding how users interface with and read on the Web, insight can be 
gained into methods of effective writing for the Web. Information should be presented in 
a concise and intuitive format (Gahran, 1998). Keep pieces short, by breaking up if 


plausible and follow these guidelines (Gahran, 1998): 


e Pare down whatever you are saying to the greatest extent that is 

appropriate. Consider using bulleted lists and highlighted key words. 

e Make sure every bit of content has a point that will matter to the user, 

and make those points obvious. 

e Break up longer stretches of text into manageable “chunks”. 

e Attempt to predict and answer all the major natural questions that your 
users will probably have about your organization or your site’s 
content. Consider dropping any content that does not focus on what 
the user probably wants to know. 

Writing for the Web is more akin to newspaper journalism than to any other form 
of distributing information, including marketing publications, magazine articles, books or 
theses. In the newspaper format, the summary or conclusions of the story are at the front 
of the story. This is done in order to immediately capture and retain the reader’s 
attention. If the reader reads these lines and is interested in the topic, they will read the 
entire article to retrieve details. This is especially true for the front page in the area 
known as “above the fold.” This prime real estate must be used to full advantage to make 
the sale by hooking the potential customer with interesting facts in order to entice them to 
purchase the paper. In the same vein, each Web page on a site must make promise of the 
information contained therein, as well as in the lower “drill-down” layers. The most 
sought after information should be summarized or encapsulated at the top of the page, in 
the writing style described above. The user will scan this page and determine if it 
contains the information needed, and read further or drill-down further for more details. 
Nielsen (1996) refers to this type of writing as the inverted pyramid style. This allows 


the page creator to identify the purpose of the site near the top of the page so the reader 


can easily see if they want to invest time at that site. 
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Creating hyperlinks in text serves to highlight that text and to provide a place so 
the user can drill-down to obtain more information, as desired. This is one of the most 
powerful features of the Web, and this is what creates the “webbing” effect of the 
Internet. This allows Web writers to split their writing into smaller, coherent pieces to 
avoid long scrolling pages (Nielsen, 1996). Scrolling will be discussed later in this 
thesis. 

In their paper, Concise, SCANNABLE, and Objective: How to Write for the Web, 
Morkes and Nielsen describe a study they conducted which started with a control version 
of a test site and modified the content into the three categories and one final category 
which combined characteristics of the three. The modified versions fared better than the 
original version containing marketing jargon or “marketese”. The concise version had 
promotional writing style but the text was much shorter, with only the more important 
information being retained. The “scannable” version also contained some marketese, but 
used bulleted lists, boldface text to highlight key words, photo captions, shorter sections 
of text, and more headings to encourage scanning. The objective version was stripped of 
| all promotional language, presenting information without exaggeration, subjective claims, 
or boasting. The combined version had shorter word count, was formatted for 
scannability, and was stripped of marketese. For overall usability, the concise style 
scored 58 percent better than the control, the scannable version was judged to be 47 
percent better, and the objective sample was 27 percent better according to the study. The 
combined version scored 124 percent better than the control version. (Morkes and 
Nielsen, 1997) This study statistically proves that investing in good writing practices 


designed specifically for the Web definitely pays dividends. A common thread between 
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conciseness, scannability, and objectivity is that each reduces the user’s cognitive load, 
which results in faster, more efficient processing of information. Concise text contains 
less information to process; scannable text calls attention to key information; and 
questioning the credibility of promotional statements seems to distract users from 
processing the meaning of that information. (Morkes and Nielsen, 1998) While military 
sites may not be selling goods, they can use these practices to communicate important 
information to users without shunning them or wasting their scarce time. 

In the past, programming Web pages required no formal training. A developer 
need only be familiar with hypertext markup language (HTML) syntax. This often 
resulted in pages with content that was lifted directly from promotional literature, not 
necessarily suited to Web documents or to a particular Web page to which it was posted. 
With usability andl readability becoming a higher level concern to Web users, the Web 
site editor will become increasingly influential in future Web site management. Three 
types of editor will emerge: editors-in-chief will keep the site focused on the customer, 
or intended audience. This editor will keep the writing true to the form discussed above. 
There will also be a second-tier editor, whose job it will be to integrate any new pages, or 
additions to pages, into the site as a whole, to retain site identity and continuity and 
validate information in relation to context. The third type of editor will be a so-called 
“content gardener’, who will regularly comb through old content on the site and either 


delete it or bring it up-to-date (i.e. keeping links current). (Gahran, 1998) 
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3. Designing for the Web 


After the writing style issue has been sufficiently covered, graphic presentation is 
the next item to be addressed regarding site or page construction. The core principle is: 
never forget for whom the page is being designed. As a general rule, do not design Web 
pages, or sites for that matter, specifically for the commanding officer (CO), chief 
executive officer (CEO), or some executive steering committee, unless they will actually 
be the only users of that site, as is the case for this thesis. The key point here is to design 
for the actual intended user of the site. 

One of the requests users continually make 1s to have faster response time when 
using the Web (Nielsen, 1997c). According to Nielsen, the basic advice regarding 
response times has been the same for almost thirty years (Nielsen, 1994): 

e 0.1 second is about the limit for having the user feel that the system is 
reacting instantaneously, meaning that no special feedback is 
necessary except to display the result. 

e 1.0 second is about the limit for the user's flow of thought to stay 
uninterrupted, even though the user will notice the delay. Normally, no 
special feedback is necessary during delays of more than 0.1 but less 
than 1.0 second, but the user does lose the feeling of operating directly 
on the data. . 

e 10 seconds is about the limit for keeping the user's attention focused 
on the dialogue. For longer delays, users will want to perform other 
tasks while waiting for the computer to finish, so they should be given 
feedback indicating when the computer expects to be done. Feedback 
during the delay is especially important if the response time is likely to 
be highly variable, since users will then not know what to expect. 

Anyone who has used the Web lately knows that sub one second or even sub ten second 
response times will not be available anytime soon, especially when dialed in over a 


modem. At times, this is even the case for those connected directly to a local area 


network (LAN). Web site developers must be cognizant of this limitation and design 


I 


accordingly. Figure 1 shows the distribution of users in relation to their connection speed 


to the Internet. 
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Figure 1. (Source: Nielsen, 1997c) 

While the ratio of users connecting with slow modems (14.4 Kbps) is shrinking in 
proportion to the number connecting with fast modems (28.8 to 33.6 Kbps), the overall 
number of modem users remains approximately constant for the period covered. With 
high speed access not being widely available until at least the year 2000 (Berst, 1997), 
speed of download must continue to be the number one concern of ie designers. 

Components that impact and delay response times the most according to a 
newspaper article from the Seattle Times Web site are (Williams, 1998): 

e Pages that appear on the screen quickly. Do not load them up with big 

graphics. 

e No sound files that start running without asking if it is wanted first. 

e No blinking, animated graphics, or dancing objects. 

e No frames dividing up the screen (frames will be discussed later in this 


thesis). 


e Keep the buttons in the same place as you move through the site. These 
buttons should be logical are well labeled. 

Graphics and images are usually the largest parts of any Web page, and as such they tend 
to use up the most bandwidth. Omitting some or all graphics from a page is the quickest 
and easiest way to decrease download time of a Web page, although some smaller, well 
chosen graphics may make the page more readable. Graphics to be placed on a Web page 
Should be chosen carefully. They eiet add value to the information that the page is being 
used to convey, not just be placed there for decorative purposes. Consider that if a page 
contained ten 100KB images, the download amount would be over one megabyte of 
information. This certainly does not bode well for those using a modem connection. 

Another tip for designers to consider is the use of alternative or the “ALT=” 
attribute. This allows the user to have a description.of an object while it is loading into 
the browser. If the transfer is interrupted or “times out”, the user will know if the item is 
something they are interested in trying harder to obtain, either by reloading the page or 
returning later to download it again. This also describes objects to those who have image 
loading turned off in their browser, or those who have browsers that do not support 
images. Values set in this attribute should be a sensible description of the object they 
represent. There are four types of images contained on Web pages; page toys, navigation 
icons, supplemental or interesting, and those that are critical for understanding the page. 
(Flavell, 1998) 

Templates are one easy way to ensure continuity of design for a Web site. 


However before implementing a template, it 1s necessary and advisable to test the 
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template for its usability. One way to test the usability of the template without involving 
the usability of the content (which should be tested separately) is to “greek” the page. 
This concept, described by Nielsen (1998), involves replacing all words on the page with 
unintelligible nonsense. This forces readers to rely on the inherent communicative 
aspects of the layout to perform the test task. In the test, each user is given a set of 
“sreeked” pages and asked to identify various parts of the page (i.e. main content 
selections for the page, page title, person responsible for the page, etc.). (Nielsen, 1998) 
This tests the overall graphic design of the page, without involving content. 

In his May 1996 Alertbox column, Jakob Nielsen, a noteworthy usability expert, 
gives his top ten recommendations for Web design: 


e Using Frames 
— Users can not bookmark the current page and return to it (the 
bookmark points to another version of the frameset). 
URLs stop working. 
Printouts become difficult. 
predictability of user actions disappears . 
e Gratuitous Use of Bleeding-Edge Technology 

— Don't try to attract users to your site by bragging about use of the 
latest web technology. } 

— Mainstream users care more about useful content and your ability 
to offer good customer service. 

— Causing the user’s system to crash while visiting your site deters 
further visits. 

— Do not use newer technologies until some experience has been 
gained with respect to the appropriate ways of using the new 
techniques. | 

e Scrolling Text, Marquees, and Constantly Running Animations 

— Never include page elements that move incessantly. 

— Moving images have an overpowering effect on the human 
peripheral vision. 

— give your user some peace and quiet to actually read the text. 

— Donot use the <BLINK> tag. 


Ly 


Complex URLs 


Users try to decode the URLs of pages to infer the structure of web 
sites. | 

Users do this because of lack of support for navigation and sense of 
location in current web browsers. 

URLs should contain human-readable directory and file names that 
reflect the nature of the information space. 

Users sometimes need to type in a URL, so try to minimize the risk 
of typos by using short names with all lower-case characters and 
no special characters (many people don't know how to type a “~” 


ie Pages 


All pages should include a clear indication of the Web site to 
which they belong. 

Remember, users may access pages directly without coming in 
through your home page. 

Every page should have a link up to your home page as well as 
some indication of where they fit within the structure of your 
information space. 


Long Scrolling Pages 


Only 10% of users scroll beyond the information that is visible on 
the screen when a page comes up. 

All critical content and navigation options should be on the top 
part of the page. 

Note added December 1997: More recent studies show that users 
are more willing to scroll now than they were in the early years of 
the Web. I still recommend minimized scrolling on navigation 
pages is still highly recommended, but it is no longer an absolute 
must. 


Lack of Navigation Support 


Don't assume that users know as much about your site as you do. 
Users need support in the form of a strong sense of structure and 
place. 

Start your design with a good understanding of the structure of the 
information space and communicate this structure explicitly to the 
user. 

Provide a site map and let users know where they are and where 
they can go. 

Good search features are needed since even the best navigation 
support will never be enough. 


Non-Standard Link Colors 


Links to pages that have not been seen by the user are blue; links to 
previously seen pages are purple or red. 
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— Do not change these colors since the ability to understand what 
links have been followed is one of the few navigational aides that 
are standard in most web browsers. 

— Consistency is key to teaching users what the link colors mean. 

e Outdated Information 

~ Budget to hire someone to maintain the information of the Web 
site as part of the team. 

— Most people would rather spend their time creating new content 
than on maintenance. | 

~ Maintenance is a cheap way of enhancing the content on your 
website since many old pages keep their relevance and should be 
linked into the new pages. 

~ Some pages are better off being removed completely from the 
server after their expiration date. 

e Overly Long Download Times 

— Traditional human factors guidelines indicate 10 seconds as the 
maximum response time before users lose interest. 

~ Jt may be acceptable to increase this limit to 15 seconds for a few 
pages, only if absolutely necessary. 

— Even Web sites with high-end users need to consider download 
times. 


Other considerations ae to include the date the material on the page was written, 
as well as the date on which the page was last updated. These enhancements will lend to 
the credibility of the page and site. 

Web sites must be designed for the lowest common denominator. This means that 
every user must be able to access the site, no matter what browser they may be using. 
Fancy applets and programs should not be included on the page for this reason, as well as 
slowing download time. Purhenner the user should not be required to install some 
plug-in just to use your site. This will turn users away, and they will not use the system. 
Any additional functionality needed or desired on the site should be done on the host’s | 


Server. 
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4. Designing for an Intranet vs. an Internet 


Just as intranets and Internets serve different purposes, so do the resident pages on 
those respective systems. As discussed earlier in this thesis, a developer must keep the 
purpose of the page in mind at all times. An intranet and its pages are generally geared 
toward a subset of users; employees familiar with the organization, its structure, 
terminology, and special circumstances. Internet site users are not as familiar with the 
organization. The tasks, type and amount of information, and bandwidth and cross- 
platform needs differ between intranet and Internet users. An intranet will contain more 
sensitive information related to the day to day running of the organization, like human 
resources information, draft reports, project progress reports, etc. This Internet site will 
generally contain public information, like customer support items or general command 
information. (Nielsen, 1997d) The same tenacity should be displayed for designing 
intranet pages as for Internet pages, with continuity of design and purpose for respective 
areas being observed. Bandwidth is usually less of a concern in the intranet environment, 
so restrictions on graphics use in that respect are relaxed, although it 1s still not advisable 


to include graphics merely for the sake of including them. 


20 


Hil. METHODOLOGY 


A. SURVEY METHODOLOGY 


After consideration of the various methods of data collection for research, surveys 
and direct observation were determined to be the best procedure for this project. Surveys 
are one method to determine how subjects react to factors being studied. Participants of 
the July/August 1998 LCIA class were given two surveys, and were directly observed 
while taking the interactive portion of the second survey. Participants of the class were a 
diverse group of flag officers with equally diverse amounts of computer use experience. 
Data gathered in this thesis is intended to provide qualitative analysis of the senior 
management user in order to make recommendations to be used in the design of user 
interfaces of apelicenon: specifically aimed at this subset of users. The nine participants 
of the LCIA class did not provide a sufficient base on which to do quantitative analysis, 


however interesting conclusions could be drawn from the information gained. 


1. General Survey 


The first survey was given in an acne to assess the amount of integration of the 
computer into the lives of senior managers. The intent of this survey was to profile the 
senior management user. By determining the level of familiarity of the target group with 
current computer technology, a baseline can be established by determining the least 
common denominator. Questions in both this survey and the subsequent survey were 


designed to be as neutral as possible to determine the true user’s perspective. 
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The 5 basic characteristics of questions and answers that are fundamental to a good 


measurement process (Fowler, 1995): 


e Questions need to be consistently understood. 

Questions need to be consistently administered or communicated to 
respondents. 

e What constitutes an adequate answer should be consistently 
communicated. 

e Unless measuring knowledge is the goal of the question, all respondents 
should have access to the information needed to answer the question 
accurately. 

e Respondents must be willing to provide the answers called for in the 
question. 


How to design questions to measure subjective states (Fowler, 1995): 


Because there are no standards against which to evaluate the 
correctness or rightness of answers, standardization of the stimulus of the 
question is particularly critical in measuring subjective states. For this 
reason, designing questions that can be administered in a consistent way 
and that mean the same thing to all respondents, to the extent possible, is 
high on the list of strategies for creating good measurement of subjective ~ 
states. 

Equally important is standardizing the response task. That means 
clearly defining the dimension or continuum respondents are to use in their 
rating task and giving them a reasonable way to replace themselves, or 
whatever else they are rating, on that continuum. 


Fowler canes the answers that come from a subjective question have no absolute 
meaning, rather they are relative. The position of the answers relative to each other 1S 
where the relevant information is found. In general, surveys should ask things that 
respondents are able to reliably report. 

The first survey included general demographic questions about user 
characteristics to determine what the subjects had access to, and what they were used to 


using. There were also some questions based on a five point Likert scale (Fowler, 1995) 
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regarding Web sites on the Internet itself, as well as questions relating to the time and 
amount of time spent on electronic mail and Web access. This survey was handed out to 


the participants on 11 August 98 and collected the next day. 


2. CEE Specific Survey 


The second survey was designed to ascertain input about the CEE Web site 
specifically. Questions were asked after the group had opportunities to use the site. The 
first section of this particular survey was a set of general questions about the site based on 
a five point Likert scale. The second part was interactive, in which participants were 
shown Web pages from two sample sets, asked questions about these pages, and directly 
observed. This portion was executed on a one-on-one basis, with the pages being 
accessed for the individual in an attempt to standardize the exercise. Participants were 
not asked to go to the site and find out answers to specific questions as in the study by 
Morkes and Nielsen (1997) on usability. This study concluded that by writing in a 
concise, objective, and scannable format, designers can greatly increase usability of a 
Web page. 

In order to assure anonymity, no names were taken and the data shall be displayed 
only in aggregate form. Appendices 1 and 2 show the general survey and the CEE 
survey, respectively, while raw summary data are shown for each respective survey in 


Appendices 3 and 4. 


B. THESIS METHODOLOGY 


The usability portion of this thesis required the aforementioned surveys. The CEE 


specific survey required the creation of two sample sets of web pages. The entire LCIA 


Web site was selected for this project. A dummy or test database was created to mirror 
the actual database. This test database was linked to the Web application using ODBC 
drivers. The original pages were then modified to interact separate from the actual site to 
avoid interference with that site’s operations. As they were modified, each file name was 
appended with the string ‘tango’ to delineate it as the pseudo-original site. Next, the 
pages for Sample Two were reworked according to principles suggested by the literature 
review on usability, most notably by Jakob Nielsen in his paper, Top Ten Mistakes in 
Web Design (Nielsen, 1996b). These file names were appended with the string ‘test’. All 
pages were affected, some more heavily than others. Major redesign took place on the 
registration page, in particular. 

The LCIA class was chosen to participate in this experiment due to their 
credentials. They were an ideal target sroup to query, since they fit the senior 
management and user criteria and they were on-site. - This contributed to the ability of the 
researcher to control the experimental conditions. 

Information gained from the participant’s answers will be compared to 
information on usability obtained in the literature review to determine if the same 
principles may apply to this elite subset of the user community. Certain standard features 
and methods have been found to be useful in increasing usability of system interfaces. 
These methods can be used as a set of guidelines to enhance usability of online systems. 
There was little available information on usability of online systems, with the exception 
of one source. Jakob Nielsen’s information on usability was used extensively in this 
thesis. Few other related readings agreed with and closely matched the primary site for 


information gathering. 
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The purpose of this thesis is to determine issues related to connection of an 
ODBC database to a Web based application and to delve into the usability of such a 
system. Information on the database portion was found in sufficient quantity through a 
literature review and direct observation of the implementation of an application on the 


CEE Wen site: 


Zo 





IV. STUDY AND FINDINGS 
A. ANALYSIS OF SURVEY OF LCIA PARTIPANTS 


1. Introduction 


The author conducted a survey of the participants of the LCIA Course as 
described in the “Survey Methodology” section of Chapter Three of this thesis. The 
survey was conducted near the end of the three-week course. This was found to be 
advantageous because, as two participants noted, they may not have had the skill set to 
answer the survey early in the course. The survey was deemed to be self-explanatory, but 
the author was available to answer any questions. An early question concerned the 
meaning of intuitive. As found in the American Heritage Dictionary, intuitive is defined 
as of, pertaining to, or arising from intuition; with intuition being the act or faculty of 
knowing without the use of rational processes; immediate cognition. Participants 
returned nine of the nine surveys given out. 

The general survey attempts to profile the type of work life the senior manager 
leads, especially in relation to using computers a computer-related products and 
applications. This survey is included as Appendix A and the data collected are listed and 
summarized in Appendix C. The CEE site specific survey asks the participants specific 
questions about the Web site for the Center for Executive Education. This survey tests 
the usability of the site in the opinions of this subset of users. The CEE survey is 


included as Appendix B, with the lists and summary of data collected is Appendix D. 
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2. General Survey 


The senior managers tested in this survey spend an average of 11.06 hours per day 
at work, ranging from ten to twelve and one-half hours per day. In addition to these work 
hours, the participants indicated they spend one to two hours daily on work related 
activities in addition to the workday hours above, averaging one and one-half hours per 
day. 

All of the respondents indicated using a computer at work, with eight of nine 
doing so at home and seven of the nine using mobile computers. All rents using 
personal computers (PCs), thirty-three percent use a Macintosh, four of nine use a 
notebook computer, and two of the respondents indicated using a palm top computer. 
Seventy-eight percent use Windows 95, forty-four percent for Windows NT Server, 
thirty-three percent for Windows NT Workstation, and none reported using Unix 
Workstation. Browser use was split fairly evenly with six of the nine using Microsoft 
Internet Explorer (MSIE) and five of the nine using Netscape browsers. Obviously some 
of the respondents use two different browsers, but obtaining preference was not the goal 
of this question. Seventy-eight percent use Microsoft Office ‘95 or 97. The SIPRNET 
secure network is used by five of the nine respondents and NIPRNET unclassified 
network is used by six out of nine participants. 

Activities performed on computers include: E-mail: and Web access or Web 
browsing being performed by all respondent: word processing and presentation graphics 
being used by eighty-nine percent of participants; financial programs are used by five out 
of nine; four responses were registered each for spread sheets and an “other” category 


which required a write-in response; one individual uses DoD applications; and sixty- 
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seven percent play games on the computer. Write-in responses were FOCIS management 
package, a program to aid in learning to play the piano, E-commerce (electronic 
commerce) and human resources applications, and financial transactions. 

The users surveyed ranged from one to two Navy electronic mail accounts and 
from zero to one private account. They receive thirty electronic mail messages each per 
day and an average of eleven attachments to the electronic mail per day, with an average 
of one of these attachments being unreadable for some reason. Four of nine participants 
spend sixteen to sixty minutes per day on average reading, generating and responding to 
electronic mail, with the remaining five spending one to three hours daily on these tasks. 

Time spent on the Internet was equally divided at forty-four percent each 
spending zero to thirty minutes and thirty to sixty minutes daily respectively, browsing or 
searching the Web for any purpose, and eleven percent spending one to three hours daily 
on these activities. Forty-four percent indicated they browse or a the Web several 
times daily, thirty-three percent once daily, and twenty-two percent classified themselves 
at less than five times per week. Seventy-eight percent spend two to ten minutes on a 
Web site once they reach it, eleven percent spend zero to two minutes, and eleven percent 
spend ten to thirty minutes at a site once they get there. Table 1 shows the ranking for 


actions taken once the respondents are finished looking at a Web site. 
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Each user was asked to rank order common web page problems which detract 


from the page’s usability the most. Table 2 below shows the overall results to this 


question. 


These data seem to confirm Nielsen’s points about Web design. 
senior managers want many of the same things as the users in Nielsen’s usability tests. 


Participants were asked to agree or disagree to the following statements on a five- 


Method O 





verall Rank 





Mean 
Follow a query-retrieved 3.00 4 
suggestion eA 


Jump to another Web site without | 1.86 
using a link 


Follow a link to another Web site {| 2.00 


Close the browser 


N/A, I do not access the Web 









] 
2 











*(Can be thrown out — no 
responses given) 





Table 1. Most Popular Actions Taken When Finished Viewing a Web Site 


= 





Detracton, | Mean | Overall Rank 
Rank 


Long download times 1 
Long scrolling pages 


Scrolling text, marquees, ads, 4.44 

constantly running animations “Sf 
4 
5 
a 


Counter intuitive links cl 


Lack of navigation support esos 2-0. oc ae 


Non-standard link colors eso a 

Poor color schemes 

Other, please write below 2.00* 2 

* Other = Broken links 

Table 2. Ranking of Most Distracting Features of Web Pages 


point Likert scale. The results are shown in Table 3. 
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It also verifies that 









IISvSICE AjpEsIUI 






dasusig: Apodwoy 


sousy Apaja]dui0y 
IdISY A[peAIUIy 










Statement: 
The complexity of a Web sites address discourages me 


o 
from visiting/using that site: jot fst 


| Computers make me more productive: = ssi | Computers make me more productive: = ssi me more productive: 

Computers increase the number of tasks I must complete to 
eee ae 
Expanded use of E-mail would make me _ more 
effective ficient itsto|af 


Table 3. Responses to General Statements about Web Sites. 





The complexity of a Web site’s address does not seem to be a factor detracting from 
respondents’ use of a Web site. Overall respondents at least generally agree that while 
computers may increase workload, they are a desirable tool for Peni work 
accomplished. 

General Meet e: were asked in which participants could write in responses. The 
first of these was to list three Web sites they like and to give the reasons why. Seven 
responses involved news related sites like CNN, the Weather Channel, the Washington 
Post site and ZDNet, while six participants pecnealy listed CEO Express. The 
reoccurring comment for the CEO Express site was that it contained good links to many 
other sites of interest. There were two references to search engines, due to the wealth of 
information possible there. Other sites that were not preferred were shunned because of 
broken links, promise but no substance, gratuitous graphics, most news sites for not being 
interactive, and search engines for being low yield. Other sites were singled out for 


requiring too much time to find the desired information (i.e. non-intuitive sites), non- 


current information, too much advertising, and sites that were not connected or 


hyperlinked. 


3. CEE Specific Survey 


In the first section of this survey, participants were asked to respond to a set of 
statements specifically regarding the CEE Web site. Responses were given on a five- 


point Likert scale as seen in Table 4. 


Statement 










sousesiq A[ojo]dur0y 






dd eSIG] A[[v41ousy 


Information on the site meets my needs: 
Information on the site is intuitive: 
Information on the site is easily readable: 


: 


e)falnlale|e 01d A[[Vs9IUd‘y 


NO 


The site address is too complex: 
The amount of page scrolling is acceptable 
The link colors are acceptable: 


EE fet foley) fre 


NO 
- 


] 
5 
s[ fifi 
4 


Table 4. Responses to Section I of the CEE Specific Survey. 





Responses to the questions regarding link colors and colors used on the CEE site included 
two outliers. Comments included an indication that the respondent could not tell by the 


colors used if they had followed a link, or not. The respondent also indicated that the 


dominance of the color blue on the CEE site was objectionable, but made no further 
suggestion. 

During the interactive or second portion of the survey the participants were shown 
two sample Web pages that were at essentially the same level of the site. The user was 
asked questions about the samples. Figures showing the set of samples for each question 
accompany the results here. 


The first set of samples refers to the LCIA page shown in Figure 2 and Figure 3. 


2 
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Next Class. 26 July - 14 August 1998 


Participant Information 


(during the course) 


Participant Information 


° Schedule in calendar form (pre-course) 
e Detaled Daily Schedule 
e Local Activities e What to expect 
e Parhcipant Biographies e Hotel informaton 
e Uploading (shanng) files (Reservation deadline: 26 June 98) 
e Downloading presentations / shared hles e Car rental 
e What to wear 
Needed Data e Orders 
e Message Center 


e Profile Assessment Forms 
e Parherpant Opinion Forms 





e Professional Biography 


About the ee a 3 
LCIA Program rel ae Registration | | 
| : | 
} 
e What this program offers Your t meus sf 
e Who should attend | 
© When and where e Course reeeraton form (please compli this form first) | 
e Costs , e Course Planning Survey “ = | 
2 Faculty | 3 Transportation Plans nl 


) a : | 
Figure 2. Sample 1 of LCIA Page (Top Portion Only). 
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I cating Citnge in the Inform 


Te” tee: 





ation Age 


The Course: 


The faculty and staff of the center are committed to profoundly improving your understanding of emerging strategic and policy issues in 
conjunction with the opportunities and constraints provided by the information age. This transformation will require significant effort on your part, 
meaningful interactons with your colleagues, and an inquisitive dialogue with the faculty 


Some functions where your presence will be expected will be scheduled in the evenings (course calendar} 





rn a ee re pe item 0 wom at lideiRISI RITA 


} ? 





Expectations © Hotel Car Rental Whatt 


‘Information Information Wear _ Center 


Figure 3. Sample 2 of LCIA Page (Entire Page). 
When asked, seventy-eight percent of respondents preferred Sample Two, with the 
remaining 22 percent preferring Sample One. Sixty-seven percent judged Sample Two to 
be both more readable and more intuitive. Furthermore; six of nine indicated that Sample 
Two makes it easier to obtain information about the LCIA Program. 
The next set of questions referred users to the samples shown in Figure 4 and 
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Leading Change in the Information Age 


Content Overview 


The focus of this program 1s to mcrease the problem solnng ability in mformation supenonty of semor defense executives in performing in ther demanding leadership 
roles. The goal is to make participants more fammbar with: 


e Development and mamtenance of sound mtegrated technology infrastructures and application systems. 
e Appkcation delivery and operation. 
e Tools and methods used to momtor and evaluate performance of mstalled warfare systems. 


The matenal 1s concentrated on the underlying pnnaples of technologies and on the ability to analyze and synthesize effectrve and flemble strateges The mtellectual 
framework consists of two components: 


e Architectures, systems, and technologies component and an mformanon pokcy, strategy, evaluation, agd operanon component. Both components aim to 
provide a comprehensive understanding of mformaton. 
e Resources and how to use them for optumal effect. 
The format used for this program 1s to provide half-day modules which use readings and lectures to drive discussion, case study analysis and prachcal exercises 


aimed at providing solutions to provocahve questions or real-life situations. It nchides a balanced mux of expenenced prachhoner and academic views Parterpants 
meet with several industry representahves to learn about new and exstng technologies. 
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Figure 4. Sample 1 of Content Overview Page (What). 
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Content Overview 


The matenalis concentrated on the underlying principles of technologies and on the ability to analyze and synthesize effective and flexible 
strategies. The intellectual framework consists of two components: 


e Architectures, systems, and technologies component and an information policy, srategy, evaluation, and operation component. Both 
components aim to provide a comprehensive understanding of information. 
e Resources and how to use them for optimal effect 
The format used for this program is to provide half-day modules which use readings and lectures to drive discussion, case study analysis and 


practical exercises aimed at prowiding solutions to provocatve questions or reablife situations. It includes a balanced mix of expenenced 
prachtioner and academic views. Participants meet with several industry representatives to leam about new and exsting technologies. 
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Figure 5. Sample 2 of Content Overview Page (About LCIA). 
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Beni eninehircent metered Sample Fwo and thought 1t was more readable, with 
the remaining eleven percent choosing Sample One. All participants, nine of nine, chose 
Sample Two as being more intuitive and making it easier to obtain information about 
cost. 

Next, participants were asked to judge which sample made it easier to navigate 
from the Content Overview page to the Course Registration Form and why. Two of nine 
preferred Sample One, six of nine preferred Sample Two, and one stated that they were 
equally bad. When asked why, respondents choosing Sample One cited one less mouse 
chck or fewer steps to get there and that it was more readily, easily available. Sample 
Two was touted for being more intuitive (two responses) and easier to read, the obvious 


selection, less scrolling, and three responses cited the registration icon, “it is easier to 


navigate from ‘buttons’ than narrative text’, and the navigation bar taking the participant 
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to “Register” whereas Sample One requires searching. The “equally bad” response was 
qualified with the indication that Sample Two had too many mouse clicks and Sample 
One was too hard to find and required scrolling. One click to the registration page would 
have been the ideal state for this respondent. 

The sample registration forms used for the next set of questions are shown in 


Figure 6 and Figure 7. 


Registration 
All nformaton provided will be used for official course purposes only 


LCIAS8-1  Feb-Mar 98 
@LCIA98-2 Jul-Aug 98 
CLCIA99-1 TBD 
CLCIA99-2. TBD 


First Name [eae 
Middle Inthal [ 
Last Name ae 
Informal Name for Nametag = 
Rakf oo” 
Service a a 
Job Title | ] 
Orgamzahon 
SsN| (See 


Course 


Personal data 


Organizational data 


Figure 6. Sample 1 of Course Registration Page (Top Portion of Page Only). 





Registration 


All information provided will be used for official course purposes only 


Course 
Class Dates Class Dates 


“LCIA98-1  Feb-Mar 98 © LCIAS9-1 TBD 
© LCIA98-2 Jul-Aug 98 “LCIAG9S-2. TBD 


Personal data 


First Name Middle Initial Last Name 

Informal Name for Nametag Rank Semice 

| [o-10 ~| [USN a 

Job Title Organization : 
SSN 


Organizational data 
Figure 7. Sample 2 of Course Registration Form (Top Portion of Page Only). 
Six of nine chose Sample Two as being easier to navigate and easier to use, three chose 
Sample One. The same margin preferred each sample respectively, with six choosing 
Sample Two and three for Sample One. Comments received include Sample Two having 
less scrolling and bolder boxes. Two of nine indicated Sample One was more readable, 
while seven of nine preferred Sample Two. One of nine chose Sample One as more 
intuitive, seven chose Sample Two, and one indicated that the choices were the same. All 
nine participants preferred the drop boxes used for the “Rank” and ‘Service” entries and 
agreed that they were easier to use. Two of nine would prefer drop boxes for the state 


entry, while seventy-eight percent would not like to see drop boxes for state entries 
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qualified with comments like, “too many entries to scroll through’, “list too long’’, and “J 
can easily type two letters.” 

One question asked which view of the “About LCIA Page” or “Content Overview 
Page” from Sample Two (Figure 5) the users preferred. The choices were the MSIE view 
and the Netscape Communicator view, with the only difference between the two being 
the font used. The MSIE version used Arial and the Netscape version used Times New 
Roman. One hundred percent chose the Arial version stating that it was larger type, even 
though no change was made to font size. One participant preferred the application’s 
buttons better in Netscape Communicator, but when asked to consider the Web page only 
and not the application execution itself, that participant chose the Arial version as well. 

Only one of nine would like to see the schedules presented differently. That user 
suggested putting a “click on a date for detailed daily schedule” note at the bottom of the 
page or on each week’s row, in addition to the note to this effect at the top of the page. 
Delineating the start time for each day in the overview, or general, schedule was also 
suggested. 

When asked what the users found most detracting about the CEE Web site, the 
most popular answers were too much scrolling noted by three respondents, and two 
respondents noted missing biographies on some of the staff and faculty members. Four 
participants made mention of having difficulty discerning which opinion forms they had 
completed and may like to be able to review previously entered answers. Two users | 
would like to see larger font size. One participant said, “I like the wealth of information 
added on Sample Two...the more the better. On the current site, it looks like more 


information is available than really is.” All responses can be seen in Appendix D. 
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4. Concluding Remarks 


Even though both samples contained the same information overall, Sample Two 
was preferred. This can be traced to the graphic design of the page. Sample Two was 
designed based on the previously discussed usability information. The pages were kept 
simple and as short as possible. The registration page was designed to resemble a paper 
form and includes more white space. According to the surveys, senior managers want 
simple, readable, intuitive, and easy to use Web pages. It is imperative that designers of 
pages and sites intended for use by senior managers follow design principles aise in 
this thesis. Interfaces must be intuitive and readable due to the lack of time these 
managers have or are willing to invest to learn how to use an interface. A site that is 
designed according to these principles will allow senior managers to use that site without 
referring to manuals or having to attend courses for instruction. This will prevent the 
Navy, and civilian institutions, from wasting money building Internet or intranet 
applications that are not used at all or not used to their fullest capacity. 

Senior managers are far more interested in the content contained on the pages than 
in graphic wizardry, or so called “bells and whistles.” This ae the necessity to 
practice good writing habits and good design practices when designing reports and 
applications. These items should be presented in a brief, encapsulated form, graphically 


if possible. 


B. ODBC 


Since senior managers are interested in the content of a page or site, it is important 


for developers to present information of interest to the target user, not just data. Data 1s 
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raw vocal, numerical, written, or observed facts about a specific topic, where as 
information can be defined as the product of processing and combining data to generate 
facts which may pertain to multiple topics (Surovchak, 1998). Figure 8 shows the 


relationship of these two terms. 


Data c—» — Information 


Figure 8. Relationship Between Data and Information (Surovchak, 1998). 

Information is something that is useful to someone when making a decision, 
whereas data is just a compilation of numbers that has not had usable knowledge 
extrapolated from it yet. ODBC becomes important here because it creates the possibility 
of creating numerous dynamic Web pages, which also creates the opportunity to create 
sites that are rich in data, but poor in information. These pages can be powerful because 
they ride on top of databases with data that is itself dynamic, but it fe important for these 
pages to present viable information. Senior managers do not want data that they must 


process themselves, that is the function of an information system. 


1. Definition 


Open database connectivity (ODBC) is a standard database access method 
developed by Microsoft Corporation. The goal of ODBC is to make it possible to access 
any data from any application, regardless of which database management system 
(DBMS) is handling the data. ODBC manages this by inserting a middle layer, called a 
database driver, between an application and the DBMS. (PC Webopaedia, 1997) This 


layer translates data queries from the application into commands that the DBMS can 
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understand. Both the application and the DBMS must implement the ODBC standard, or 
be ODBC compliant, in order for this system to work. This means that an application 
must be capable of issuing ODBC commands and the DBMS must be capable of 
responding to them. Figure 9 shows the relationship of applications, data stores, and 


ODBC. 
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Client Application 


ODBC Driver Manager 





Figure 9. ODBC Model. (Adapted from Forta, 1998.) 


2. History 


ODBC was created in an attempt to allow Microsoft Excel to access diverse data 
stores. In April 1988, a Microsoft employee proposed a model that used database drivers 
to isolate the native data types of different database applications. This model, using 


application program interfaces (APIs), would allow client software to communicate with 


any message store. To access a particular data store, all that would be required was a 
driver designed specifically for that data store. Four other companies were working on a 
similar project during this time, and between 1988 and 1992 these companies worked 
together to shape one specification. After undergoing several name changes, ODBC 1.0 


was released in September 1992. (Forta, 1998) 


3. How ODBC Works 


ODBC facilitates access and manipulation of information by end-users in a 
manner congruent with their roles within an organization. The ODBC standard from 
Microsoft provides a transparent database independent interface between desktop based 
productivity tools — spreadsheets, work processing software, graphics packages, report 
writers, etc. and data sources such as relational DBMS engines. (Kingsley) The ODBC 
interface allows maximum interoperability between applications and back-end database 
management systems. An application developer can develop, compile, and ship an 
application without targeting a specific DBMS product. Users can then add modules 
called database drivers that link the application to their choice of DBMS. 

Referring again to Figure 9, the primary purpose of the ODBC driver manager is 
to load and track drivers that actually implement ODBC function calls. The drivers 
perform ODBC function calls and interact with the data source. To obtain this 
functionality, the driver manages the communication protocols between the application 
and the data source when the application requests to connect to the data source. Once this 
connection is established, a driver is capable of handling requests to the DBMS that are 


made by the application. The driver also provides the necessary data translations to and 
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from the data source, and returns the results to the application. Furthermore, a driver will 
also standardize the error codes that are returned to it so that an application that is using 


multiple drivers will not have to manage discrepancies. (Signore, 1995) 


Figure 10 below shows the basic flow of control for ODBC applications: 
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Allocate Environment 


Allocate Connect Handle 


ee | 
| Connect to Server 


Allocate Statement Handle 


l 


Statement Processing 





& 
SQL 
Processing Receiving Parts 
Free Statement Handle 
Disconnect from Server 
Terminating 


Free Connection Handle 


Free Environment 


Figure 10. Basic Flow of Control for ODBC Applications. (Signore, 1995) 
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ODBC uses standard query language (SQL) to interact with databases. 
This language, designed initially by IBM, has become the industry standard for 
manipulating data in a database. Many of the programs available today have 
query builders or “wizards” that help the user to graphically define a query by 


dragging and dropping items they wish to manipulate. 


4. Implementation 


Many commercial-off-the-shelf (COTS) products support the ODBC standard. 
There are a plethora of products from which to choose, including spreadsheets, word 
processors, graphics programs, and many database management systems. For the CEE 
Web site, the following products were used: Microsoft Access, Cold Fusion, and 
Windows NT Server. . 

Microsoft Access is a scalable relational database and DBMS that integrates data 
from spreadsheets and other databases. It can share information over intranets and the 
Internet. Access is part of the Office 97 Professional suite of software from Microsoft, 
which is specified in the Commander in Chief, Pacific Fleet (CINCPACFLT) message 
regarding Information Technology for fhe 21* Century or “IT-21” (CINCPACFLT, 
1997). Access is ODBC compliant, and the included Wizards make it easy to create 
databases and applications. 

IT-21 is a Navy initiative agreed upon by CINCPACFLT and Commander in 
Chief, Atlantic Fleet (CINCLANTFLT) to standardize commands under these areas of 
responsibility to a consistent set of software and hardware products. This strategy will 


allow reduction of training costs and compatibility concerns. The goal of IT-21 is to link 
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all U.S. forces, and eventually even our allies, together in a network that enables voice, 
video, and data transmissions from a single desktop PC. This will allow warfighters to 
easily exchange information that is classified or unclassified, and tactical or non-tactical. 
To do this, a system must be built to industry standards, using commercial off-the-shelf 
technology (COTS), devoid of systems that do not communicate, in a client-server 
environment that allows the user to pull information needed in a way that is seamless to 
the user in the field. (Clemins, 1998) This means Access 97 should be available at many 
commands as a basic part of the software inventory, and no additional purchase would be 
necessary. 

Allaire Corporation’s Cold Fusion is a rapid application development tool that 
enables a designer to integrate browser, server, and database technologies into Web-based 
applications. It allows creation of dynamic Web applications and interactive Web sites. 
Cold Fusion uses any ODBC compliant database as a data source, can generate and 
receive electronic mail using simple mail transfer protocol (SMTP)/post office protocol 
(POP), includes file management capabilities, supports structured query language (SQL) 
queries, and can be managed and administered from any Web browser located anu | 
with Internet access. 

Windows NT Server, from Microsoft, is a server operating system for building 
and hosting Web based applications. This allows a server to reply to hypertext transfer 
protocol (HTTP) requests from browsers, perform security and user authenticatian 
encrypt Web pages, and perform server side processing. 

Using ODBC is merely a matter of ensuring products that can communicate with 


DBMSs are in compliance with the ODBC standard. In the case of the CEE Web site, all 
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of the products are ODBC compliant. If the right software is installed on the server, a 
developer can design and implement databases. If proper ODBC drivers are installed, 
developers can register newly created databases with the Cold Fusion software. All of 
these administrative tasks are aided by graphic user interfaces that allow the user to 
browse directories and access online help functions. Cold Fusion acts as an intermediary 
between the Web page server and the ODBC driver manager. This allows a developer to 
write Web pages that include SQL eatement: that can bring data out of the database to be 
presented, or processed in some way. This processing might include adding . updating 
data to the database, or manipulating the data in some way to produce information for 
presentation to a user, whether this user is a senior manager or not. Each user is a client 


who should be presented useful information in a usable format. 
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V. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 


Knowledge of usability of Web based information tools by senior managers holds 
value for designers of Web pages and sites, as well as the users of those applications. By 
knowing what to expect and demand, users can shape the future of these designs, and by 
knowing what users want and desire in a Web based application, designers can provide 
systems that are usable and effective. 

The literature review found, and the surveys and observation proved, that content 
is indeed king to users of Web applications. It is irrelevant if a developer follows all of 
the usability guides; if the content on a page is lacking, the user will look elsewhere. For 
this con it is imperative that writers aim documents specifically at the electronic 
medium of the World Wide Web. The importance of keeping the information on a site 
correct and current can not be over emphasized, as it lends to the quality and credibility 
of the site. 

Web linking was an issue that presented itself during this study. If information 
eee an item is available elsewhere on the Web, a link can and should be used. This 
takes advantage of one of the strengths of the World Wide Web. Benefits of this practice 
include allowing someone else to own and maintain their information, reducing workload 
on an organization’s technical staff, saving valuable time, and reducing redundancy and © 
sources of data error, inconsistency, inaccuracy, timeliness, etc. However, before 
installing a link on their site, developers must review the content of the information 


contained in the page that has been linked to ensure accuracy, to the best of their ability. 
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A set of decision criteria should be set up to determine whether to include links whose 
information is not completely accurate. Including these links on a site may increase 
richness of a site, but may also sacrifice credibility of the “referring” site. The 
aforementioned decision criteria must address this issue. Broken or incorrect links are 
one supporting argument for continuous “gardening” or maintaining of Web sites, and is 
highly recommended. 

Another point to take uae from this thesis is that the purpose of the Web site 
must remain clear to the designers and maintainers. The target audience for the Web 
application must be identified and queried as to what they find useful. The design must 
be tested on actual users to determine if their needs are being met. 

Simplicity is essential. Pages free from complex graphics or blinking or scrolling 
text will not distract the user when viewing the page, and it will help ensure reasonable 
download times. By following the guidelines set forth in this thesis, a designer can 
assemble a Web site that keeps its users in mind by designing specifically for them. 
Internet or intranet Web sites have the potential to provide easily accessible information 
to those who need it, in the most timely manner possible. 

Information must be extrapolated from data contained in databases. Connection 
of a database to a Web application is key in obtaining this goal of providing useful 
information. Developers must simply be aware of the challenge and purchase products 
that - ODBC cabeetiaate so that a DBMS and an application requesting data can 


communicate to provide this useful information. 
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B. RECOMMENDATIONS 


The information obtained from the literature review and the insight gained from 


the surveys and observation lead to several suggestions. These concluding statements 


should provide the stimulus for changes supported by the collated data. 


The main suggestion entails migrating the CEE Web site to a modified version of 


the model presented in Sample Two of the project created for this thesis. Suggested 


modifications include: 


: 


Amount of graphics on CEE site good, but add no more; 

Move navigation buttons to tops of pages, and reduce their size; 

Use standard link colors; 

Decrease the amount of scrolling required per page without sacrificing 
content; 

Provide security for privacy act type data (beyond the cone of this thesis); 
Ensure all content on the site 1s current and accurate (i.e. biographies are 
available to participants of the course); 

Opinion forms: make it more obvious which forms have been completed by 
making the colors more obvious for links that have been followed or including 
check boxes on the main opinion form selection page for those forms that 


have been completed. 


Suggested Further Studies 


The lack of literature in this area suggests a fertile area to be probed by future 


researchers. Additional bodies of data, large enough to be statistically significant should 


be gathered and analyzed. Continued surveying with the current surveys, or a similar 
updated survey of this type, would provide further insight into the senior manager as a 
user of these systems. Additional questions to be added to the survey may be to ascertain 
the average senior managers’ years of experience using a computer to establish a baseline 
for this group, and another to attempt to determine how advanced this subset of users are, 
on average. 

A study relating the effects of linked information should be performed. This 
study should attempt to determine whether users prefer links to information that may not 
be entirely accurate, or not link at all. It could also Ties the effects of linked 
information on the referring Web site. 

Another interesting project would be to continue to survey participants of the 
courses taught by the Center for Executive Education. Identical surveys could be given 


before and after attendance of the course to determine the effectiveness of the courses. 
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APPENDIX A. GENERAL WEB SURVEY 


Please read the following before continuing... 


Information gained by this survey will be used in a thesis aimed at creating better Web 
interface design for senior managers. Accurate information is essential to developing an 
accurate profile for senior managers. Feedback will also be integrated into the CEE Web 
site. The information obtained will also allow us to profile the typical user at the senior 
management level. The information obtained in this survey is and will remain 
anonymous; no names will be taken or associated in any way to the information gained. 


This survey should take approximately 10 to 15 minutes to complete. 


Thank you for taking the time to participate. 
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Please answer the following questions to the best of your knowledge: 


1. How many hours per day, on average, do you spend at work? ae 


2. How many hours per day, on average, do you spend on work related activities in 
addition to the hours noted 1n Question 1? = 


3. J use a computer at (please check all that apply): OX Home OwWork Ol Mobile 


4. Iuse (please check all that apply): 


O aPC O) a Macintosh LO) a Notebook CL] a Palm Top 
LJ Windows 95 CL) Windows NT Server 1 Windows NT Workstation 
(1) Netscape Browser UW Microsoft (MS) Internet'Explorer Browser CL) Unix Workstation 


LI MS Office 95 or 97 UJ Secure network (SIPRNET) 0) Unclassified machines (NIPRNET) 


5. What types of activities do you perform on computers (please check all that apply, 
whether at work or at home): 


O E-mail O Presentation Graphics (i.e. Power Point, etc.) 

O Word Processing O DoD Applications (Financial, Supply, Travel 
Manager, etc.) 

O1 Web Access / Browsing 4 Spread sheet (Excel, Lotus 1-2-3, etc.) 


O Games 1 Financial Programs (Quicken, MS Money, etc.) 
oe ee 
E-Mail: 
6. With respect to E-mail, how many accounts do you have: 
Navy account(s) [| Private account(s) (i.e. AOL, MSN, etc.) [| 


7. Onan average workday, how many E-mail messages do you receive (not counting 
leave)? [| How many of these messages have attachments? 
How many of these are you not able read (for any reason)?[_ | 


8. Approximately how much time per day, on average, do you spend generating, 
reading, and responding to E-mail? 


(10-15 minutes 016-60 minutes 011-3 hours WU More than 3 hours 
Internet: 
9. In total, approximately how much time per day, on average, do you spend on the 
Internet (i.e. Browsing or searching the web) for any purpose? 
C} 0-30 minutes C1 30-60 minutes O1-3 hours QO More than 3 hours 
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10. Approximately how often do you Browse or Search the Web? 


CL) Several times daily CL] Once daily O Less than five times per week 
1 Less than ten times per month 11 Never 
11. When you get to a Web site, how long do you stay there? 


O) 0-2 minutes © 2-10 minutes OF 10-30 minutes 0 More than 30 minutes 


12. When you are finished looking at a Web site, do you (rank in order, with 1 being most 
used action): 


[_| Follow a query-retrieved suggestion (i.e. go back to a search engine and follow a 
link from there) 
[_] Jump to another Web site without using a link 


[_] Follow a link to another Web site [_] Close the browser [_] N/A, I do not access the Web 


13. Regarding the graphical user interface used on Web sites you visit, what do you find 
most detracting? (Rank in order, with 1 being most detracting) 


[_] Long download times [_] Counter intuitive links L_] Non-standard link colors 
{_] Long scrolling pages [_] Lack of navigation support L_} Poor color schemes 
[_] Scrolling text, marquees, ads, constantly running animations _[_] Other, please write below 


| QO 
a ) ° 
Q @ = 
YO Te PTT] 
2 | g Z| es S 
@ 2 o => @ 
OF peel | ee 
> qr - s Za 
ge = go > 
o © a ‘e 
e & 
14. The complexity of a Web site’s address discourages me from 
yisihim gsi) (atest Cem mean ene ence eee eet etn 2 Oo 0 O 0 DO 
15, Computers makemesmore prodticinie 2.2.47. -.--. 7 ss eee erences Fes eee (os | nl ed 
16. Computers increase the number of tasks I must complete. 
to the extent that I experience increased workload..................6. BB ae ee 
17. Expanded use of E-mail would make me more effective/efficient... OD [~ OF OF © 


18. List 3 Web sites you like and give the reason(s) you like each one (you do not have to use the exact 


address) (NOTE: If you can not think of 3 sites, please list 3 features you like about Web sites): 


a7 


19. List 3 Web sites you do not like and give the reason(s) for each (you do not have to use the exact 
address) (NOTE: If you can not think of 3 sites, please list 3 features you like about Web sites): 


Note: This survey came out to three total pages in finished form. Due to formatting differences 
here it increased to four pages. 
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APPENDIX B. CEE SITE SPECIFIC WEB SURVEY 


Section I - General 


9013 Aja}ajduiog 
d013Y A][eisuaH 


Regarding the CEE Web site, please respond to each 
of the following: 


a) 
o 
= 
o 
-— 
= 
= 
o 
<2) 
= 
ge 
-— 
© 
o 





Information on the site is relevant: 
Information on the site is accurate: 
Information on the site is complete: 
Information on the site is current: 
Information on the site meets my needs: 
Information on the site is intuitive: 
Information on the site is easily readable: 
The site address is too complex: 

9. The amount of page scrolling 1s acceptable: 
10. The link colors are acceptable: 

11. The colors used throughout the site are acceptable: 
12. The amount of graphics is acceptable: 

13. I would like to see more graphics: 

14. Download times are acceptable: 

15. Navigation support 1s acceptable: 

16. Navigation through the site is intuitive: 


Ce Ot aatag tO 
E) SaaS ages agaee)| eo apaeee 
Se ee) ee) See eye) Sie) ©) el 


CE OE Boe) fb aoa ee oO oe 
Eee eee a) eee) ema ee 


Section II - Interactive 


Please answer the next set of questions regarding the following graphical user 
interfaces: 


Go to the LCIA Page: 

17. Which of the samples do you prefer? 1 2 Whichismorereadable? 1 2 
Which is more intuitive? 1 2 

18. Which sample makes it easier to obtain information about the LCJA Program? 1 2 


Go to the Content Overview Page (What or About LCIA): 


19. Which of the samples do you prefer? 1 2 Whichismorereadable? 1 2 
Which is more intuitive? 1 2 


20. Which sample makes it easier to obtain information about cost? 1 2 
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Ooo0o00gpoopoGoooog 


Go to the Course Registration Form: 


21. Which sample page made it easier to navigate from the “Content Overview Page” to 
the “Course Registration Form”? 1 2 Why? 


22. Which sample page 1s easier to navigate? 1 2 
23. Which sample registration form is easier to use? 1 2 


24. Which form layout do you prefer? 1 2 Whichis more readable? 1 2 
Which is more intuitive? 1 2 


25. For the ‘Rank’ and ‘Service’ boxes, which is easier to use? 1 2 
Which do you prefer? 1 2 


26. Would you prefer “drop boxes” for the state entries? Yes No 





27. Which view of the “About LCIA” (or “Content Overview Page”) from sample 2 do 
you prefer? 


As viewed in MS Internet Explorer As viewed in Netscape 
Communicator 


28. Would you like to see the schedules presented differently? Yes No 
If Yes, how? 
29. Regarding the graphical user interface used on the CEE Web site, what do you find 


most detracting? 


30. What changes would you make to the CEE Web site (additions or deletions)? Please 
specify below. 
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APPENDIX C. GENERAL WEB SURVEY DATA 
Summary N=9 
Please answer the following questions to the best of your knowledge: 
1. How many hours per day, on average, do you spend at work? 


Raw answers: 12, 11, 12 (60), 10, 12.5, 12, 10, 10, 10 
Range: Low: 10 High: 12.5 
Mean: 11.06 hours per day 


(Special Note: one participant responded with an answer of 60 hours. When 
compiling the data, a reasonable assumption was made to divide this by 5 for an 
average of 12 hours worked per day. With that response thrown out, the Mean 
becomes 10.94 hours per day.) 


2. How many hours per day, on average, do you spend on work related activities in 
addition to the hours noted in Question 1? 


Raw answers: 2, 2, 1, 1, 1, 1to2, 1, 2, 2 

Range: Low: 1 High: 2 

Mean: 1.5 hours per day 
(Special Note: one participant responded with a range of 1 to 2 hours. When 
compiling the data, this response was averaged and 1.5 was used to calculate the 
mean for this question. With this response thrown out, the Mean remains 1.5 hours 
per day.) 
3. I use a computer at (please check all that apply): | Home Work Mobile 


Responses: 8 “9 a 
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4. I use (please check all that apply): Responses: 


A PC 

A Macintosh 

A Notebook 

A Palm Top 

Windows 95 

Windows NT Server 

Windows NT Workstation 
Netscape Browser 

Microsoft (MS) Internet Explorer Browser 
Unix Workstation 

MS Office 95 or 97 

Secure network (SIPRNET) 
Unclassified machines (NIPRNET) 


DMI “A N&W bh ~ IN & W 


S. What types of activities do you perform on computers (please check all that apply, 
whether at work or at home): 


Responses: 
E-mail | 9 
Word Processing 8 
Web Access / Browsing 9 
Presentation Graphics (1.e. Power Point, etc.) 8 
DoD Applications (Financial, Supply, Travel: Manager, etc.) 1 
Spread sheet (Excel, Lotus 1-2-3, etc.) 4 
Financial Programs (Quicken, MS Money, etc.) 5 
Games 6 
Other (please write in): 4 


e FOCIS — Management Package 
e Learn to play piano 

e E-Commerce/HR Application 

e Financial transactions 

E-Mail: 


6. With respect to E-mail, how many accounts do you have: 
Navy account(s) 


Raw answers: 2, 1, 1, 1, 2, 2, 2, 2, 2 


Range: Low: 1 High: 2 
Mean: 1.67 Navy E-mail accounts per person 


62 


Private account(s) (1.e. AOL, MSN, etc.) 


Raw answers: 0, 1, 1, 1, 0, 1, 1, 0, 1 
Range: Low: 0 High: 1 
Mean: 0.67 Private E-mail accounts per person 


7. Onan average workday, how many E-mail messages do you receive (not counting 
leave)? 


Raw answers: 45, 50, 10, 30, 50, 12, 30, 30, 10 
Range: Low: 10 — High: 50 
Mean: 29.67 Messages per person per day 


How many of these messages have attachments? 


Raw answers: 10, 5, 20, 15, 30, 3, 10, 5, 3 
Range: Low: 3 High: 30 
Mean: 11.22 Attachments per person per day 


How many of these are you not able read (for any reason)? 
Raw answers: 3, 1, 1, 1, 1, 0, 1, 1, 0 


Range: Low: 0 High: 3 
Mean: 1.00 Unreadable attachments per person per day 


8. Approximately how much time per day, on average, do you spend generating, 
reading, and responding to E-mail? 


Responses: 
0-15 minutes 
16-60 minutes 4 
1-3 hours 5 


More than 3 hours 


Internet: 
9. In total, approximately how much time per day, on average, do you spend on the 
Internet (i.e. Browsing or searching the web) for any purpose? 


Responses: 
0-30 minutes 4 
30-60 minutes 4 
1-3 hours 1 


More than 3 hours 


10. Approximately how often do you Browse or Search the Web? 


Responses: 
Several times daily 4 
Once daily 3 
Less than five times per week 2 
Less than ten times per month 
Never 


11. When you get to a Web site, how long do you stay there? 


Responses: 
0-2 minutes 1 
2-10 minutes 7 
10-30 minutes 1 


More than 30 minutes 


12. When you are finished looking at a Web site, do you (rank in order, with 1 being most 
used action): 
Key (in bold): 
A. Follow a query-retrieved suggestion (i.e. go back to a search engine and 
follow a link from there) 
B. Jump to another Web site without using a link 
C. Follow a link to another Web site 
D. Close the browser 
E. N/A, I do not access the Web 


Outcome: 


Method 1 Zits 5 J Mean | Overall Rank | 
Rank | 


Follow a query-retrieved S '2 Som 2 4 1/3 |3.00 |4 
aggsion Pe el 
Jump to another Web site without 

ngs 


[Follow a link to another Web site |_| 1_ 2 [2.00 
ceeteboms se e+ tte 
; 


*(Can be thrown out — no responses) 


(Special Note: the above mean rank includes the responses where the participant 
only checked one block vice ranking them in the order as requested. The checked 
boxes were counted as #1, with no other responses being posted. Throwing these 
responses out had no effect on the overall ranking. Mean rank for the data if the 3 
participants’ data in question are left out are as follows: A = 3.00, B = 2.00, C = 
2.167, and D = 2.83) 


64 
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13. Regarding the graphical user interface used on Web sites you visit, what do you find 
most detracting? (Rank in order, with 1 being most detracting) 


Outcome: 


Detractor Mean | Overall! Rank 
Rank 


Fe a es ‘Long downloadtimes 1 Salat. 22st eae see 
Long scrolling pages s+ Sop eS 2.78 
Scrolling text, marquees, ads, 4.44 
aie SPP E PET eT 
‘Counterintutve inks [4 [4 [3 [4 ]2 [216 [4 [4 [367 [a 
Lack of navigation suppor [516 [4[3 [4 [4 [1 [3-15 [389 [3 
[Non-standard link colors [6 |7 [6 [7 [3 [6 [315 [6 [589 [7 


[Poor color schemes ___—~—~+(|7 [5/7 [81717 |4 16 |7 [64a [8 
(Other, please write below | | | [2] | | | | [2.00 2 


Other = Broken links 


aaisy Ajajajduioa 
daisy AT[eIauaH 
[enon 
sa1d3esiq A]]eisuayH 
daisesiq Ajajajduiog 


14. The complexity of a Web site’s address discourages 

me Mom Visitine/ Using that. Slie.............2.....:... ae 3 5 ] 
15. Computers make me more productive: tee ee 3 66 

16. Computers increase the number of tasks I must complete 

to the extent that I experience increased workload.............. 3 3 1 

17. Expanded use of E-mail would make me more 

eliectivie/ ClnC Keni et ceca. s: eckt ns eee. eae a een ae ae | Zz “I 
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18. List 3 Web sites you like and give the reason(s) you like each one (you do not have to 
use the exact address) (NOTE: If you can not think of 3 sites, please list 3 features 
you like about Web sites): 


e Snap! News, top stories door = quick, comprehensive info 
e The Weather Channel = quick, comprehensive info 

e ZDnet Anchor Desk = good access to IT news 

e NUWCNET - work home page with useful info 

e Dictionary & thesaurus — useful and easy to use 

e Dilbert — mood lifting, easy 

e Travel Manager 

e Business News/Stock quotes 

e Military home pages 

e CEQOexpress — Meta site 

e Alta Vista — go find it 

e WashingtonPost.com — best news on the Net 

e CEOexpress.com — quick info/quick links to related sites 
e DefenseLink.mil — quick info/quick links to related sites 
e Anchordesk.com — quick info/quick links to related sites 
e CEOexpress 

e AOL 

e CNN 

e Cnn.com — latest news 

e CEOexpress.com — best meta links 

e Wired.com — good tech stuff 

e ...CEO...— great homepage to jump off from 

e ...schwab... - investments/trading 

e ...quicken... - stock quotes/graphs 

e CEOexpress — excellent links = 

e BUPERSnet — potential high for info pertinent to me 

e HOTBOT - access to good info 
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19. List 3 Web sites you do not like and give the reason(s) for each (you do not have to 


use the exact address) (NOTE: If you can not think of 3 sites, please list 3 features 
you like about Web sites): 


Search engines = low yield 

Travel — very hard to find what I want 

E-bird — bad scrolling 

Broken links 

Promise, but no substance 

Gratuitous graphics 

Most news sites — Not interactive 

IETF.ORG -— too technical 

Airlines — too much time required 

Shopping — Eddie Bauer — too hard, not user-friendly 
Interlotto.com — too complicated 

NY Times — have to subscribe 

FlagWeb — not real current 

(LIKE) — Not too much advertising content on the screen 
(LIKE) — Ease of use — Intuitive 

(LIKE) — Connected/Hyperlinked | 

BUPERSnet — potential high, but frequently does not deliver 
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APPENDIX D. CEE SITE SPECIFIC WEB SURVEY DATA 


Summary N=9 


Section I - General 


© 
© 
= 
o 
ot 
= 
< 
2 
1 ei 
© 
ge 
= 
© 
O 


said y Ajajajduog 
s018y AT[eI9UayH 
saidesiq, Ajajoydwog 


Regarding the CEE Web site, please respond to each 
of the following: 





1. Information on the site 1s relevant: 5 4 

2. Information on the site is accurate: 3 4 2 

3. Information on the site is complete: 3 4 1 1 
4. Information on the site 1s current: 3 Z 3 1 

5. Information on the site meets my needs: 3 6 

6. Information on the site is intuitive: 4 aD 1 

7. Information on the site is easily readable: 6 3 

8. The site address is too complex: 1 1 3 + 
9. The amount of page scrolling is acceptable: 1 a2 1 
10. The link colors are acceptable: 2 5 1 1 
11. The colors used throughout the site are acceptable: Zz 6 1 
12. The amount of graphics is acceptable: 1 4 3 1 
13. I would like to see more graphics: 1 See 

14. Download times are acceptable: 3 5 1 

15. Navigation support is acceptable: 5 3 1 

16. Navigation through the site is intuitive: 3 4 2 


Section II - Interactive 


Please answer the next set of questions regarding the following graphical user 
interfaces: | 


Go to the LCIA Page: Sl S2 

17. Which of the samples do you prefer? Z 7 
Which is more readable? 3 6 
Which is more intuitive? 3 6 


18. Which sample makes it easier to obtain information about the LCIA Program? 
3 6 
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Go to the Content Overview Page (What or About LCIA): S1 S2 


19. Which of the samples do you prefer? 1 8 
Which is more readable? 1 8 
Which is more intuitive? 0 9 

20. Which sample makes it easier to obtain information about cost? 0 9 

Go to the Course Registration Form: S1 S2 


21. Which sample page made it easier to navigate from the “Content Overview Page” 
to the “Course Registration Form’? 2 6 
Why? - One answer: Equally Bad 

Other comments: 

e 2- Easier to read and more intuitive 


e 2- Obvious selection 

e Equally bad — No direct link 

e 2- Less scrolling/more intuitive 

e 2 - Registration icon 

e 2-—-Nav[igation] Bar takes you to “Register” - #1 requires search 

e 1 —-—One less click 

e 2-FEasier to navigate from “buttons” than [from] narrative text . 

e 1-—Seems to be fewer steps to get there. More readily, easily available 
22. Which sample page is easier to navigate? 3 6 


Other comments: _ 
e 2- Less scrolling 


23. Which sample registration form is easier to use? 3 6 
Other comments: 
e Bolder boxes 


24. Which form layout do you prefer? | 5 6 
Which is more readable? Zz i 
Which is more intuitive? One “Same” 1 a 

25. For the ‘Rank’ and ‘Service’ boxes, which 1s easier to use? 0 9 
Which do you prefer? 0 9 

Yes No 

26. Would you prefer “drop boxes” for the state entries? 2 7 


Other comments: 

e Too many entries to scroll through 
e §6List too long 

e I can easily type two letters 
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27. Which view of the “About LCIA” (or “Content Overview Page’’) from sample 2 do 
you prefer? 


As viewed in MS Internet Explore As viewed in Netscape Communicator 
9 0 
Note: The only difference between these pages was the font (Arial vs. Times New 
Roman) 
Comments: 


e MSIE - bigger type 

e (Note: One of the participants liked the buttons better in Netscape 
Communicator — even though the font size was smaller — when asked directly to 
concentrate only on the page content , without considering the browser 
application itself, the participant liked the MSIE version better. 

e Font (Note: This participant liked the font in MSIE better) 


Yes No 
28. Would you like to see the schedules presented differently? 1 8 
If Yes, how? 


e Maybe put “click on a date for detailed daily schedule” at bottom or on each 
week (Note: . this would be in addition to the statement at the top of the page) 
e Delineate start time for each day in overview schedule 


29. Regarding the graphical user interface used on the CEE Web site, what do you find 
most detracting? 

Comments: 

Like simple stuff, no “bells and whistles” 

Like input, needs to be intuitively obvious 

Scrolling — would be nice if it fit on one page 

Small type size/too much scrolling | 

I don’t find anything distracting. Colors, etc. do not bother me — it’s content 

Disjointed — seems a little “hodgepodge” — thrown together 

Emphasis on Blue color — would like to see more variation 

Missing [some] bios in faculty section 

After entering the information on the participant opinion forms, would like to be 

locked out so that I know that J have already voted on that day’s activities 
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30. What changes would you make to the CEE Web site (additions or deletions)? Please 


specify below. 


Comments: 


Bios not there for all presenters 

Like opinion forms 

Like to go in, get info, get out 

More obvious color choices to which evaluation have been completed (Note: 
hard to tell which links have already been followed — especially on the machines 
used in the experiment, which were laptops.) 

Why list Topsail — It is ‘under construction’? (You click on it and it tells you 
there is no class scheduled for 1998) 

Larger type/More intuitive links (HTTP) 

I’m happy with it 

Use Nav bars as in Sample 2 

Uniformity/same feel 

Click on a bullet to next link 

Opinion forms — Automatically go to today’s date 

Maintain history of previous opinion form answers 

Decrease scrolling 

Put more [heavily] used choices at or near top of page 

[Put] Early Bird on external links 

I like the wealth of info added on Sample 2...The more the better. On the 
current site, it looks like more info is available tan really is. 
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